public inbox for [email protected]
help / color / mirror / Atom feedRe: A qsort template
3+ messages / 3 participants
[nested] [flat]
* Re: A qsort template
@ 2022-04-02 20:20 Justin Pryzby <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: Justin Pryzby @ 2022-04-02 20:20 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Thomas Munro <[email protected]>; Peter Geoghegan <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Sat, Apr 02, 2022 at 06:41:30PM +0700, John Naylor wrote:
> On Sat, Apr 2, 2022 at 5:27 PM Thomas Munro <[email protected]> wrote:
> > Reproduced locally, using the same few lines from the cluster.sql
> > test. I'll try to dig more tomorrow...
>
> Thanks! Unfortunately I can't reproduce locally with clang 13/gcc 11,
> with -Og or -O2 with CFLAGS="-fsanitize=undefined,alignment" ...
Like Thomas just said, I had to use:
CFLAGS="-Og -fsanitize=undefined,alignment -fno-sanitize-recover=all
I'm a couple few steps out of my league here, but it may be an issue with:
commit 4ea51cdfe85ceef8afabceb03c446574daa0ac23
Author: Robert Haas <[email protected]>
Date: Mon Jan 19 15:20:31 2015 -0500
Use abbreviated keys for faster sorting of text datums.
This is enough to avoid the crash, which might be a useful hint..
@@ -4126,22 +4126,23 @@ copytup_cluster(Tuplesortstate *state, SortTuple *stup, void *tup)
/*
* set up first-column key value, and potentially abbreviate, if it's a
* simple column
*/
+ stup->isnull1 = false;
if (state->indexInfo->ii_IndexAttrNumbers[0] == 0)
return;
original = heap_getattr(tuple,
state->indexInfo->ii_IndexAttrNumbers[0],
state->tupDesc,
&stup->isnull1);
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: A qsort template
@ 2022-04-02 20:37 Thomas Munro <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Thomas Munro @ 2022-04-02 20:37 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: John Naylor <[email protected]>; Peter Geoghegan <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Sun, Apr 3, 2022 at 8:20 AM Justin Pryzby <[email protected]> wrote:
> @@ -4126,22 +4126,23 @@ copytup_cluster(Tuplesortstate *state, SortTuple *stup, void *tup)
> + stup->isnull1 = false;
Looks like I might have failed to grok the scheme for encoding null
into SortTuple objects. It's clearly uninitialised in some paths,
with a special 0 value in datum1. Will need to look more closely with
more coffee...
^ permalink raw reply [nested|flat] 3+ messages in thread
* [PATCH v45 6/8] Fix a few problems in index build progress reporting.
@ 2026-03-27 15:50 Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Álvaro Herrera @ 2026-03-27 15:50 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
6 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 68a42de0889..1c8226c6d67 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1181,7 +1181,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index de7182a85a9..cfd33f73f48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,6 +715,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -760,6 +763,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1276,7 +1280,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1438,6 +1443,12 @@ index_create_copy(Relation heapRelation, bool concurrently,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1528,7 +1539,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3001,6 +3012,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3011,7 +3023,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3062,6 +3075,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3819,7 +3833,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 83edab38760..c9e22e95515 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1233,6 +1233,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 56a064ef444..a03238304a0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -71,6 +71,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -148,7 +149,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--cldir6umqisr673d
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v45-0007-Error-out-any-process-that-would-block-at-REPACK.patch"
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-03-27 15:50 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-04-02 20:20 Re: A qsort template Justin Pryzby <[email protected]>
2022-04-02 20:37 ` Thomas Munro <[email protected]>
2026-03-27 15:50 [PATCH v45 6/8] Fix a few problems in index build progress reporting. Álvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox