public inbox for [email protected]
help / color / mirror / Atom feed[18] clarify the difference between pg_wchar, wchar_t, and Unicode code points
29+ messages / 5 participants
[nested] [flat]
* [18] clarify the difference between pg_wchar, wchar_t, and Unicode code points
@ 2024-04-15 23:40 Jeff Davis <[email protected]>
0 siblings, 1 reply; 29+ messages in thread
From: Jeff Davis @ 2024-04-15 23:40 UTC (permalink / raw)
To: pgsql-hackers
I'm not sure I understand all of the history behind pg_wchar, but it
seems to be some blend of:
(a) Postgres's own internal representation of a decoded character
(b) libc's wchar_t
(c) Unicode code point
For example, Postgres has its own encoding/decoding routines, so (a) is
the most obvious definition. When the server encoding is UTF-8, the
internal representation is a Unicode code point, which is convenient
for the builtin and ICU providers, as well as some (most? all?) libc
implementations. Other encodings have different represenations which
seem to favor the libc provider.
pg_wchar is also passed directly to libc routines like iswalpha_l()
(see pg_wc_isalpha()), which is depending on definition (b). We guard
it with:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
to ensure that the pg_wchar is representable in the libc's wchar_t
type. As far as I can tell this is still no guarantee of correctness;
it's just a sanity check. I didn't find an obviously better way of
doing it, however.
When using ICU, we also pass a pg_wchar directly to ICU routines, which
depends on definition (c), and can lead to problems like:
https://www.postgresql.org/message-id/[email protected]
The comment at the top of pg_regc_locale.c explains some of the above,
but not all. I'd like to organize this a bit better:
* a new typedef for a Unicode code point ("codepoint"? "uchar"?)
* a no-op conversion routine from pg_wchar to a codepoint that would
assert that the server encoding is UTF-8 (#ifndef FRONTEND, of course)
* a no-op conversion routine from pg_wchar to wchar_t that would be a
good place for a comment describing that it's a "best effort" and may
not be correct in all cases
We could even go so far as to make the pg_wchar type not implicitly-
castable, so that callers would be forced to convert it to either a
wchar_t or a code point.
Tom also suggested here:
https://www.postgresql.org/message-id/360857.1701302164%40sss.pgh.pa.us
that we don't necessarily need to use libc at all, and I like that
idea. Perhaps the suggestions above are a step in that direction, or
perhaps we can skip ahead?
I intend to submit a patch for the July CF. Thoughts?
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: [18] clarify the difference between pg_wchar, wchar_t, and Unicode code points
@ 2024-04-18 19:18 Peter Eisentraut <[email protected]>
parent: Jeff Davis <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Peter Eisentraut @ 2024-04-18 19:18 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; pgsql-hackers
On 16.04.24 01:40, Jeff Davis wrote:
> I'm not sure I understand all of the history behind pg_wchar, but it
> seems to be some blend of:
>
> (a) Postgres's own internal representation of a decoded character
> (b) libc's wchar_t
> (c) Unicode code point
>
> For example, Postgres has its own encoding/decoding routines, so (a) is
> the most obvious definition.
(a) is the correct definition, I think. The other ones are just
occasional conveniences, and occasionally wrong.
> When using ICU, we also pass a pg_wchar directly to ICU routines, which
> depends on definition (c), and can lead to problems like:
>
> https://www.postgresql.org/message-id/[email protected]
That's just a plain bug, I think. It's missing the encoding check that
for example pg_strncoll_icu() does.
> The comment at the top of pg_regc_locale.c explains some of the above,
> but not all. I'd like to organize this a bit better:
>
> * a new typedef for a Unicode code point ("codepoint"? "uchar"?)
> * a no-op conversion routine from pg_wchar to a codepoint that would
> assert that the server encoding is UTF-8 (#ifndef FRONTEND, of course)
> * a no-op conversion routine from pg_wchar to wchar_t that would be a
> good place for a comment describing that it's a "best effort" and may
> not be correct in all cases
I guess sometimes you really want to just store an array of Unicode code
points. But I'm not sure how this would actually address coding
mistakes like the one above. You still need to check the server
encoding and do encoding conversion when necessary.
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 6/7] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 544dd0d6e3d..13284f1d708 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1455,6 +1460,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1545,7 +1556,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3018,6 +3029,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3028,7 +3040,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3079,6 +3092,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3836,7 +3850,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d89be1b9533..4fd10a5e7b3 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1164,9 +1164,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3694,6 +3691,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4b37d92a49e..8d1e8a20998 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 144e241ebd3..6c7f0fee115 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -152,7 +153,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
filename=v42-0007-Teach-snapshot-builder-to-skip-transactions-running-.patch
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v43 6/7] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 98d38dc5229..1d28202de34 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1455,6 +1460,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1545,7 +1556,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3018,6 +3029,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3028,7 +3040,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3079,6 +3092,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3836,7 +3850,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 36950aea780..065316ee866 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1162,9 +1162,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3616,6 +3613,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 836eedb8ed5..1c4a887da13 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 144e241ebd3..6c7f0fee115 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -152,7 +153,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--nzintiedl6o4kcyp
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v43-0007-Teach-snapshot-builder-to-skip-transactions-runn.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v44 09/10] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 2c3058ba10d..018e9f7b013 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1153,9 +1153,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3582,6 +3579,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--gwom7bl7ogtszo4k
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v44-0010-Teach-snapshot-builder-to-skip-transactions-runn.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v43 6/7] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 98d38dc5229..1d28202de34 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1455,6 +1460,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1545,7 +1556,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3018,6 +3029,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3028,7 +3040,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3079,6 +3092,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3836,7 +3850,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 36950aea780..065316ee866 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1162,9 +1162,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3616,6 +3613,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 836eedb8ed5..1c4a887da13 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 144e241ebd3..6c7f0fee115 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -152,7 +153,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--nzintiedl6o4kcyp
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v43-0007-Teach-snapshot-builder-to-skip-transactions-runn.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v44 09/10] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 2c3058ba10d..018e9f7b013 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1153,9 +1153,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3582,6 +3579,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--gwom7bl7ogtszo4k
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v44-0010-Teach-snapshot-builder-to-skip-transactions-runn.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 17118f2fe76..cb06c52c1d4 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1180,7 +1180,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b1e05d56a1e..89aad2f7b77 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -714,6 +714,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -759,6 +762,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1275,7 +1279,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1454,6 +1459,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1544,7 +1555,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3017,6 +3028,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3027,7 +3039,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3078,6 +3091,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3835,7 +3849,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fe419ef808e..9aaa8e95389 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1162,9 +1162,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3505,6 +3502,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 34209bd1393..ad5c1238e09 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1230,6 +1230,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 3426087b445..3778e9b7f07 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -65,6 +65,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -146,7 +147,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 17118f2fe76..cb06c52c1d4 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1180,7 +1180,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b1e05d56a1e..89aad2f7b77 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -714,6 +714,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -759,6 +762,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1275,7 +1279,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1454,6 +1459,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1544,7 +1555,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3017,6 +3028,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3027,7 +3039,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3078,6 +3091,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3835,7 +3849,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fe419ef808e..9aaa8e95389 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1162,9 +1162,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3505,6 +3502,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 34209bd1393..ad5c1238e09 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1230,6 +1230,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 3426087b445..3778e9b7f07 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -65,6 +65,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -146,7 +147,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 6/7] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 544dd0d6e3d..13284f1d708 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1455,6 +1460,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1545,7 +1556,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3018,6 +3029,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3028,7 +3040,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3079,6 +3092,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3836,7 +3850,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d89be1b9533..4fd10a5e7b3 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1164,9 +1164,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3694,6 +3691,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4b37d92a49e..8d1e8a20998 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 144e241ebd3..6c7f0fee115 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -152,7 +153,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
filename=v42-0007-Teach-snapshot-builder-to-skip-transactions-running-.patch
^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: [PATCH] Fix incorrect Spanish translation and remove obsolete FIXME comments
@ 2026-03-18 08:20 John Naylor <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: John Naylor @ 2026-03-18 08:20 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: VASUKI M <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Feb 20, 2026 at 4:55 PM Daniel Gustafsson <[email protected]> wrote:
>
> > On 20 Feb 2026, at 10:52, VASUKI M <[email protected]> wrote:
>
> > While reviewing the Spanish translation file (src/backend/po/es.po),
> > I found an incorrect translation in the message:
>
> Thanks for your submission, all translation related fixes should be posted to
> the pgsql-translators@ mailinglist.
>
> https://www.postgresql.org/list/pgsql-translators/
Further to that, the commitfest is not the right place for it, either.
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v46 5/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--mdsv5bdgfjjj6d77
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v46-0006-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v47 7/9] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--2fxmamo6mu2qbxgv
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v47-0008-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v48 5/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--qfkt2ktdpcfeypib
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v48-0006-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v49 4/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--3n3vqr5y2jhknrqj
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v49-0005-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v50 4/8] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--brnevsqjnuzpyok4
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v50-0005-support-repacking-tables-with-exclusion-constrai.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v51 04/10] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--r2slln3zpmilwu22
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v51-0005-invert-meaning-of-index_create-flag-bit.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v52 04/10] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--gp2pyozrd5pweboh
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v52-0005-invert-meaning-of-index_create-flag-bit.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v53 2/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 26 +++++++++++++++++++++-----
src/backend/catalog/toasting.c | 3 ++-
src/include/catalog/index.h | 4 +++-
5 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index c52c0a6023d..ebd41176b94 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1184,7 +1184,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..36712b1bcc2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_SUPPRESS_PROGRESS:
+ * don't report progress during the index build.
+ *
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,8 +1453,16 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * The current callers do not need to report progress: if 'concurrently'
+ * is true, there is no build at all to report about; and otherwise the
+ * index build is a sub-command of REPACK, and the current progress
+ * reporting infrastructure does not allow two commands to report their
+ * progress at the same time.
+ */
if (concurrently)
- flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
+ flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT |
+ INDEX_CREATE_SUPPRESS_PROGRESS;
/*
* Now create the new index.
@@ -1538,7 +1551,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3022,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3033,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3085,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3843,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..dcee536fd3f 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..9f538dac798 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--qr3jlalmmcpkiodg
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v53-0003-Rename-cluster.c-h-repack.c-h.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v45 6/8] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--cldir6umqisr673d
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v45-0007-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v46 5/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--mdsv5bdgfjjj6d77
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v46-0006-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v47 7/9] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--2fxmamo6mu2qbxgv
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v47-0008-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v48 5/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--qfkt2ktdpcfeypib
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v48-0006-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v49 4/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b86ad73c626..bac8d527ae5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4a2d21915b1..25e8c1945c5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--3n3vqr5y2jhknrqj
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v49-0005-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v50 4/8] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--brnevsqjnuzpyok4
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v50-0005-support-repacking-tables-with-exclusion-constrai.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v52 04/10] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--gp2pyozrd5pweboh
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v52-0005-invert-meaning-of-index_create-flag-bit.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v53 2/7] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 26 +++++++++++++++++++++-----
src/backend/catalog/toasting.c | 3 ++-
src/include/catalog/index.h | 4 +++-
5 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index c52c0a6023d..ebd41176b94 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1184,7 +1184,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..36712b1bcc2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_SUPPRESS_PROGRESS:
+ * don't report progress during the index build.
+ *
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,8 +1453,16 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * The current callers do not need to report progress: if 'concurrently'
+ * is true, there is no build at all to report about; and otherwise the
+ * index build is a sub-command of REPACK, and the current progress
+ * reporting infrastructure does not allow two commands to report their
+ * progress at the same time.
+ */
if (concurrently)
- flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
+ flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT |
+ INDEX_CREATE_SUPPRESS_PROGRESS;
/*
* Now create the new index.
@@ -1538,7 +1551,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3022,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3033,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3085,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3843,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..dcee536fd3f 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..9f538dac798 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--qr3jlalmmcpkiodg
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v53-0003-Rename-cluster.c-h-repack.c-h.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v51 04/10] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38ef683d4c7..60fb7051830 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1183,7 +1183,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e418d67e8e4..4b8ed2c7660 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1448,6 +1453,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1538,7 +1549,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3009,6 +3020,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3019,7 +3031,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3070,6 +3083,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3827,7 +3841,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..51b27a8c71c 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -332,7 +332,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index cba379810c7..932924c13e0 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1231,6 +1231,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index ed9e4c37d27..7ebe4f0bd87 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--r2slln3zpmilwu22
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v51-0005-invert-meaning-of-index_create-flag-bit.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v45 6/8] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--cldir6umqisr673d
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v45-0007-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
end of thread, other threads:[~2026-03-27 15:50 UTC | newest]
Thread overview: 29+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-04-15 23:40 [18] clarify the difference between pg_wchar, wchar_t, and Unicode code points Jeff Davis <[email protected]>
2024-04-18 19:18 ` Peter Eisentraut <[email protected]>
2026-03-16 12:53 [PATCH 6/7] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH v44 09/10] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH v43 6/7] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH v44 09/10] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH v43 6/7] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-16 12:53 [PATCH 6/7] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
2026-03-18 08:20 Re: [PATCH] Fix incorrect Spanish translation and remove obsolete FIXME comments John Naylor <[email protected]>
2026-03-27 15:50 [PATCH v49 4/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v50 4/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v46 5/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v45 6/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v51 04/10] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v47 7/9] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v48 5/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v52 04/10] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v47 7/9] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v46 5/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v52 04/10] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v45 6/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v51 04/10] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v48 5/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v49 4/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v53 2/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v50 4/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
2026-03-27 15:50 [PATCH v53 2/7] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox