public inbox for [email protected]help / color / mirror / Atom feed
Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() 5+ messages / 4 participants [nested] [flat]
* Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() @ 2022-03-02 12:48 Bharath Rupireddy <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Bharath Rupireddy @ 2022-03-02 12:48 UTC (permalink / raw) To: Nitin Jadhav <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On Wed, Mar 2, 2022 at 5:41 PM Nitin Jadhav <[email protected]> wrote: > > Hi, > > I have noticed that the CHECKPOINT_REQUESTED flag information is not > present in the log message of LogCheckpointStart() function. I would > like to understand if it was missed or left intentionally. The log > message describes all the possible checkpoint flags except > CHECKPOINT_REQUESTED flag. I feel we should support this. Thoughts? I don't think that's useful. Being in LogCheckpointStart (CreateCheckPoint or CreateRestartPoint) itself means that somebody has requested a checkpoint. Having CHECKPOINT_REQUESTED doesn't add any value. I would suggest removing the CHECKPOINT_REQUESTED flag as it's not being used anywhere instead CheckpointerShmem->ckpt_flags is used as an indication of the checkpoint requested in CheckpointerMain [1]. If others don't agree to remove as it doesn't cause any harm, then, I would add something like this for more readability: if ((((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags) & CHECKPOINT_REQUESTED)) { do_checkpoint = true; PendingCheckpointerStats.m_requested_checkpoints++; } [1] /* * Detect a pending checkpoint request by checking whether the flags * word in shared memory is nonzero. We shouldn't need to acquire the * ckpt_lck for this. */ if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags) { do_checkpoint = true; PendingCheckpointerStats.m_requested_checkpoints++; } Regards, Bharath Rupireddy. ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() @ 2022-03-03 00:39 Kyotaro Horiguchi <[email protected]> parent: Bharath Rupireddy <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Kyotaro Horiguchi @ 2022-03-03 00:39 UTC (permalink / raw) To: [email protected]; +Cc: [email protected]; [email protected] At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy <[email protected]> wrote in > On Wed, Mar 2, 2022 at 5:41 PM Nitin Jadhav > <[email protected]> wrote: > > > > Hi, > > > > I have noticed that the CHECKPOINT_REQUESTED flag information is not > > present in the log message of LogCheckpointStart() function. I would > > like to understand if it was missed or left intentionally. The log > > message describes all the possible checkpoint flags except > > CHECKPOINT_REQUESTED flag. I feel we should support this. Thoughts? > > I don't think that's useful. Being in LogCheckpointStart > (CreateCheckPoint or CreateRestartPoint) itself means that somebody > has requested a checkpoint. Having CHECKPOINT_REQUESTED doesn't add > any value. Agreed. > I would suggest removing the CHECKPOINT_REQUESTED flag as it's not > being used anywhere instead CheckpointerShmem->ckpt_flags is used as > an indication of the checkpoint requested in CheckpointerMain [1]. If Actually no one does but RequestCheckpoint() accepts 0 as flags. Checkpointer would be a bit more complex without CHECKPOINT_REQUESTED. I don't think it does us any good to get rid of the flag value. > others don't agree to remove as it doesn't cause any harm, then, I > would add something like this for more readability: if (((volatile CheckpointerShmemStruct *) - CheckpointerShmem)->ckpt_flags) + CheckpointerShmem)->ckpt_flags) & CHECKPOINT_REQUESTED)) I don't particularly object to this, but I don't think that change makes the code significantly easier to read either. regards. -- Kyotaro Horiguchi NTT Open Source Software Center ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() @ 2022-03-03 01:27 Michael Paquier <[email protected]> parent: Kyotaro Horiguchi <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Michael Paquier @ 2022-03-03 01:27 UTC (permalink / raw) To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected] On Thu, Mar 03, 2022 at 09:39:37AM +0900, Kyotaro Horiguchi wrote: > At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy <[email protected]> wrote in >> I don't think that's useful. Being in LogCheckpointStart >> (CreateCheckPoint or CreateRestartPoint) itself means that somebody >> has requested a checkpoint. Having CHECKPOINT_REQUESTED doesn't add >> any value. > > Agreed. Exactly my impression. This would apply now to the WAL shutdown code paths, and I'd suspect that the callers of CreateCheckPoint() are not going to increase soon. The point is: the logs already provide some contexts for any of those callers so I see no need for this additional information. > Actually no one does but RequestCheckpoint() accepts 0 as flags. > Checkpointer would be a bit more complex without CHECKPOINT_REQUESTED. > I don't think it does us any good to get rid of the flag value. I'd rather keep this code as-is. -- Michael Attachments: [application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc) download ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() @ 2022-03-03 03:04 Kyotaro Horiguchi <[email protected]> parent: Michael Paquier <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Kyotaro Horiguchi @ 2022-03-03 03:04 UTC (permalink / raw) To: [email protected]; +Cc: [email protected]; [email protected]; [email protected] At Thu, 3 Mar 2022 10:27:10 +0900, Michael Paquier <[email protected]> wrote in > On Thu, Mar 03, 2022 at 09:39:37AM +0900, Kyotaro Horiguchi wrote: > > At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy <[email protected]> wrote in > >> I don't think that's useful. Being in LogCheckpointStart > >> (CreateCheckPoint or CreateRestartPoint) itself means that somebody > >> has requested a checkpoint. Having CHECKPOINT_REQUESTED doesn't add > >> any value. > > > > Agreed. > > Exactly my impression. This would apply now to the WAL shutdown code > paths, and I'd suspect that the callers of CreateCheckPoint() are not > going to increase soon. The point is: the logs already provide some > contexts for any of those callers so I see no need for this additional > information. > > > Actually no one does but RequestCheckpoint() accepts 0 as flags. > > Checkpointer would be a bit more complex without CHECKPOINT_REQUESTED. > > I don't think it does us any good to get rid of the flag value. > > I'd rather keep this code as-is. I fail to identify the nuance of the phrase, so just for a clarification. In short, I think we should keep the exiting code as-is. regards. -- Kyotaro Horiguchi NTT Open Source Software Center ^ permalink raw reply [nested|flat] 5+ messages in thread
* [PATCH v50 4/8] Fix a few problems in index build progress reporting. @ 2026-03-27 15:50 Álvaro Herrera <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw) First, index_build() should not update the progress when being driven by REPACK, because the progress reporting infractructure cannot handle status of two commands at the same time. So far, REPACK with the CONCURRENTLY option neglected this problem altogether, but even the existing REPACK wasn't consistent enough: even if the 'progress' variable in repack_index() was false, it didn't pass the value to index_build(). Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which does the same thing. --- src/backend/bootstrap/bootstrap.c | 2 +- src/backend/catalog/heap.c | 3 ++- src/backend/catalog/index.c | 22 ++++++++++++++++++---- src/backend/catalog/toasting.c | 3 ++- src/backend/commands/indexcmds.c | 1 + src/include/catalog/index.h | 4 +++- 6 files changed, 27 insertions(+), 8 deletions(-) diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 38ef683d4c7..60fb7051830 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -1183,7 +1183,7 @@ build_indices(void) heap = table_open(ILHead->il_heap, NoLock); ind = index_open(ILHead->il_ind, NoLock); - index_build(heap, ind, ILHead->il_info, false, false); + index_build(heap, ind, ILHead->il_info, false, false, false); index_close(ind, NoLock); table_close(heap, NoLock); diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index 5748aa9a1a9..ae6b7cda3dd 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation) /* Initialize the index and rebuild */ /* Note: we do not need to re-establish pkey setting */ - index_build(heapRelation, currentIndex, indexInfo, true, false); + index_build(heapRelation, currentIndex, indexInfo, true, false, + true); /* We're done with this index */ index_close(currentIndex, NoLock); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index e418d67e8e4..4b8ed2c7660 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid, * already exists. * INDEX_CREATE_PARTITIONED: * create a partitioned index (table must be partitioned) + * INDEX_CREATE_REPORT_PROGRESS: + * update the backend's progress information during index build. + * constr_flags: flags passed to index_constraint_create * (only if INDEX_CREATE_ADD_CONSTRAINT is set) * allow_system_table_mods: allow table to be a system catalog @@ -760,6 +763,7 @@ index_create(Relation heapRelation, bool invalid = (flags & INDEX_CREATE_INVALID) != 0; bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0; bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0; + bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0; char relkind; TransactionId relfrozenxid; MultiXactId relminmxid; @@ -1276,7 +1280,8 @@ index_create(Relation heapRelation, } else { - index_build(heapRelation, indexRelation, indexInfo, false, true); + index_build(heapRelation, indexRelation, indexInfo, false, true, + progress); } /* @@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently, stattargets[i].isnull = isnull; } + /* + * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If + * 'concurrently' is true, there is no build at all. Otherwise the index + * build is a sub-command of REPACK. The current infrastructure does not + * allow two commands to report their progress at the same time. + */ if (concurrently) flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT; @@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId, indexInfo->ii_BrokenHotChain = false; /* Now build the index */ - index_build(heapRel, indexRelation, indexInfo, false, true); + index_build(heapRel, indexRelation, indexInfo, false, true, true); /* Roll back any GUC changes executed by index functions */ AtEOXact_GUC(false, save_nestlevel); @@ -3009,6 +3020,7 @@ index_update_stats(Relation rel, * * isreindex indicates we are recreating a previously-existing index. * parallel indicates if parallelism may be useful. + * progress indicates if the backend should update its progress info. * * Note: before Postgres 8.2, the passed-in heap and index Relations * were automatically closed by this routine. This is no longer the case. @@ -3019,7 +3031,8 @@ index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo, bool isreindex, - bool parallel) + bool parallel, + bool progress) { IndexBuildResult *stats; Oid save_userid; @@ -3070,6 +3083,7 @@ index_build(Relation heapRelation, RestrictSearchPath(); /* Set up initial progress report status */ + if (progress) { const int progress_index[] = { PROGRESS_CREATEIDX_PHASE, @@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId, /* Initialize the index and rebuild */ /* Note: we do not need to re-establish pkey setting */ - index_build(heapRelation, iRel, indexInfo, true, true); + index_build(heapRelation, iRel, indexInfo, true, true, progress); /* Re-allow use of target index */ ResetReindexProcessing(); diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c index 4aa52a4bd25..51b27a8c71c 100644 --- a/src/backend/catalog/toasting.c +++ b/src/backend/catalog/toasting.c @@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid, BTREE_AM_OID, rel->rd_rel->reltablespace, collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0, - INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL); + INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0, + true, true, NULL); table_close(toast_rel, NoLock); diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index cba379810c7..932924c13e0 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate, flags |= INDEX_CREATE_PARTITIONED; if (stmt->primary) flags |= INDEX_CREATE_IS_PRIMARY; + flags |= INDEX_CREATE_REPORT_PROGRESS; /* * If the table is partitioned, and recursion was declined but partitions diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index ed9e4c37d27..7ebe4f0bd87 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel, #define INDEX_CREATE_IF_NOT_EXISTS (1 << 4) #define INDEX_CREATE_PARTITIONED (1 << 5) #define INDEX_CREATE_INVALID (1 << 6) +#define INDEX_CREATE_REPORT_PROGRESS (1 << 7) extern Oid index_create(Relation heapRelation, const char *indexRelationName, @@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo, bool isreindex, - bool parallel); + bool parallel, + bool progress); extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot); -- 2.47.3 --brnevsqjnuzpyok4 Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v50-0005-support-repacking-tables-with-exclusion-constrai.patch" ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-03-27 15:50 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-03-02 12:48 Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart() Bharath Rupireddy <[email protected]> 2022-03-03 00:39 ` Kyotaro Horiguchi <[email protected]> 2022-03-03 01:27 ` Michael Paquier <[email protected]> 2022-03-03 03:04 ` Kyotaro Horiguchi <[email protected]> 2026-03-27 15:50 [PATCH v50 4/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox