From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Rahila Syed <rahila.syed@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>
Cc: David Fetter <david@fetter.org>
Cc: Pavan Deolasee <pavan.deolasee@gmail.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Subject: Re: monitoring CREATE INDEX [CONCURRENTLY]
Date: Fri, 5 Apr 2019 16:46:53 +0200
Message-ID: <ef6a6757-c36a-9e81-123f-13b19e36b7d7@2ndquadrant.com> (raw)
In-Reply-To: <20190402183132.GA15653@alvherre.pgsql>
References: <20190402183132.GA15653@alvherre.pgsql>
It seems we can easily extend this to also monitor REINDEX
[CONCURRENTLY]. Attached is a quick patch.
For the concurrently part, we currently don't have any phases defined
for the index swap and drop, but maybe we can just skip that initially.
What happens if we don't have those?
It might be nice to have a column in the view not only for the table OID
but also the index OID. That is obviously not so useful for CREATE
INDEX but more useful for REINDEX. I haven't looked into how adding
that would work.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From cae47efe485c3c6115c1970b9c922ded1353d270 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Fri, 5 Apr 2019 16:41:27 +0200
Subject: [PATCH] Report progress of REINDEX operations
This uses the same infrastructure that the CREATE INDEX progress
reporting uses.
---
src/backend/catalog/index.c | 8 ++++++++
src/backend/commands/indexcmds.c | 19 +++++++++++++++++--
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.cindex 2ed7fdb021..686b3c8260 100644--- a/src/backend/catalog/index.c+++ b/src/backend/catalog/index.c@@ -3286,12 +3286,18 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
heapId = IndexGetRelation(indexId, false);
heapRelation = table_open(heapId, ShareLock);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,+ heapId);+
/*
* Open the target index relation and get an exclusive lock on it, to
* ensure that no one else is touching this particular index.
*/
iRel = index_open(indexId, AccessExclusiveLock);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,+ iRel->rd_rel->relam);+
/*
* The case of reindexing partitioned tables and indexes is handled
* differently by upper layers, so this case shouldn't arise.
@@ -3442,6 +3448,8 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
errdetail_internal("%s",
pg_rusage_show(&ru0))));
+ pgstat_progress_end_command();+
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.cindex 348d543297..7efb4c9e15 100644--- a/src/backend/commands/indexcmds.c+++ b/src/backend/commands/indexcmds.c@@ -2873,6 +2873,11 @@ ReindexRelationConcurrently(Oid relationOid, int options)
heapRel = table_open(indexRel->rd_index->indrelid,
ShareUpdateExclusiveLock);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,+ RelationGetRelid(heapRel));+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,+ indexRel->rd_rel->relam);+
/* Choose a temporary relation name for the new index */
concurrentName = ChooseRelationName(get_rel_name(indexId),
NULL,
@@ -2967,7 +2972,9 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* DefineIndex() for more details.
*/
- WaitForLockersMultiple(lockTags, ShareLock, false);+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_1);+ WaitForLockersMultiple(lockTags, ShareLock, true);
CommitTransactionCommand();
forboth(lc, indexIds, lc2, newIndexIds)
@@ -3009,7 +3016,9 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* for more details.
*/
- WaitForLockersMultiple(lockTags, ShareLock, false);+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_2);+ WaitForLockersMultiple(lockTags, ShareLock, true);
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -3057,6 +3066,8 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, false);
CommitTransactionCommand();
@@ -3128,6 +3139,7 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* index_drop() for more details.
*/
+ // FIXME: progress update here
WaitForLockersMultiple(lockTags, AccessExclusiveLock, false);
foreach(lc, indexIds)
@@ -3150,6 +3162,7 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* Drop the old indexes.
*/
+ // FIXME: progress update here
WaitForLockersMultiple(lockTags, AccessExclusiveLock, false);
PushActiveSnapshot(GetTransactionSnapshot());
@@ -3225,6 +3238,8 @@ ReindexRelationConcurrently(Oid relationOid, int options)
MemoryContextDelete(private_context);
+ pgstat_progress_end_command();+
return true;
}
--
2.21.0
Attachments:
[text/plain] 0001-Report-progress-of-REINDEX-operations.patch (4.1K, ../ef6a6757-c36a-9e81-123f-13b19e36b7d7@2ndquadrant.com/2-0001-Report-progress-of-REINDEX-operations.patch)
download | inline diff:
From cae47efe485c3c6115c1970b9c922ded1353d270 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Fri, 5 Apr 2019 16:41:27 +0200
Subject: [PATCH] Report progress of REINDEX operations
This uses the same infrastructure that the CREATE INDEX progress
reporting uses.
---
src/backend/catalog/index.c | 8 ++++++++
src/backend/commands/indexcmds.c | 19 +++++++++++++++++--
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.cindex 2ed7fdb021..686b3c8260 100644--- a/src/backend/catalog/index.c+++ b/src/backend/catalog/index.c@@ -3286,12 +3286,18 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
heapId = IndexGetRelation(indexId, false);
heapRelation = table_open(heapId, ShareLock);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,+ heapId);+
/*
* Open the target index relation and get an exclusive lock on it, to
* ensure that no one else is touching this particular index.
*/
iRel = index_open(indexId, AccessExclusiveLock);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,+ iRel->rd_rel->relam);+
/*
* The case of reindexing partitioned tables and indexes is handled
* differently by upper layers, so this case shouldn't arise.
@@ -3442,6 +3448,8 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
errdetail_internal("%s",
pg_rusage_show(&ru0))));
+ pgstat_progress_end_command();+
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.cindex 348d543297..7efb4c9e15 100644--- a/src/backend/commands/indexcmds.c+++ b/src/backend/commands/indexcmds.c@@ -2873,6 +2873,11 @@ ReindexRelationConcurrently(Oid relationOid, int options)
heapRel = table_open(indexRel->rd_index->indrelid,
ShareUpdateExclusiveLock);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,+ RelationGetRelid(heapRel));+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,+ indexRel->rd_rel->relam);+
/* Choose a temporary relation name for the new index */
concurrentName = ChooseRelationName(get_rel_name(indexId),
NULL,
@@ -2967,7 +2972,9 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* DefineIndex() for more details.
*/
- WaitForLockersMultiple(lockTags, ShareLock, false);+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_1);+ WaitForLockersMultiple(lockTags, ShareLock, true);
CommitTransactionCommand();
forboth(lc, indexIds, lc2, newIndexIds)
@@ -3009,7 +3016,9 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* for more details.
*/
- WaitForLockersMultiple(lockTags, ShareLock, false);+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_2);+ WaitForLockersMultiple(lockTags, ShareLock, true);
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -3057,6 +3066,8 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,+ PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, false);
CommitTransactionCommand();
@@ -3128,6 +3139,7 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* index_drop() for more details.
*/
+ // FIXME: progress update here
WaitForLockersMultiple(lockTags, AccessExclusiveLock, false);
foreach(lc, indexIds)
@@ -3150,6 +3162,7 @@ ReindexRelationConcurrently(Oid relationOid, int options)
* Drop the old indexes.
*/
+ // FIXME: progress update here
WaitForLockersMultiple(lockTags, AccessExclusiveLock, false);
PushActiveSnapshot(GetTransactionSnapshot());
@@ -3225,6 +3238,8 @@ ReindexRelationConcurrently(Oid relationOid, int options)
MemoryContextDelete(private_context);
+ pgstat_progress_end_command();+
return true;
}
--
2.21.0
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: peter.eisentraut@2ndquadrant.com, alvherre@2ndquadrant.com, rahila.syed@2ndquadrant.com, andres@anarazel.de, robertmhaas@gmail.com, simon@2ndquadrant.com, pgsql-hackers@lists.postgresql.org, david@fetter.org, pavan.deolasee@gmail.com, Langote_Amit_f8@lab.ntt.co.jp
Subject: Re: monitoring CREATE INDEX [CONCURRENTLY]
In-Reply-To: <ef6a6757-c36a-9e81-123f-13b19e36b7d7@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox