agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
96+ messages / 5 participants
[nested] [flat]
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v10 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 88a68a4697..aaf955458c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -18153,6 +18153,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -18164,9 +18167,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx), false);
parentTblId = get_partition_parent(RelationGetRelid(partedTbl), false);
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--mvpLiMfbWzRoNl4x--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--AA9g+nFNFPYNJKiL--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..a478c13990 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17521,6 +17521,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17532,9 +17535,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--O5XBE6gyVG5Rl6Rj--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8687e9a97c..865ab6c2e9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17493,6 +17493,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17504,9 +17507,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index b0f02bc1f6..2a7d9b463c 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--FsscpQKzF/jJk6ya--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions
@ 2020-11-25 23:34 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Justin Pryzby @ 2020-11-25 23:34 UTC (permalink / raw)
This is required for restoring clustered parent index, which is marked INVALID
until indexes have been built on all its child tables, and it's prohibited to
CLUSTER ON an INVALID index
See also: 8cff4f5348d075e063100071013f00a900c32b0f
---
src/backend/commands/tablecmds.c | 6 +++---
src/bin/pg_dump/common.c | 8 ++++++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..eb56890311 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17497,6 +17497,9 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
table_close(idxRel, RowExclusiveLock);
}
+ /* make sure we see the validation we just did */
+ CommandCounterIncrement();
+
/*
* If this index is in turn a partition of a larger index, validating it
* might cause the parent to become valid also. Try that.
@@ -17508,9 +17511,6 @@ validatePartitionedIndex(Relation partedIdx, Relation partedTbl)
Relation parentIdx,
parentTbl;
- /* make sure we see the validation we just did */
- CommandCounterIncrement();
-
parentIdxId = get_partition_parent(RelationGetRelid(partedIdx));
parentTblId = get_partition_parent(RelationGetRelid(partedTbl));
parentIdx = relation_open(parentIdxId, AccessExclusiveLock);
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 1a261a5545..cdfba058fc 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -429,6 +429,12 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
attachinfo[k].parentIdx = parentidx;
attachinfo[k].partitionIdx = index;
+ /*
+ * We want dependencies from parent to partition (so that the
+ * partition index is created first)
+ */
+ addObjectDependency(&parentidx->dobj, index->dobj.dumpId);
+
/*
* We must state the DO_INDEX_ATTACH object's dependencies
* explicitly, since it will not match anything in pg_depend.
@@ -446,6 +452,8 @@ flagInhIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
*/
addObjectDependency(&attachinfo[k].dobj, index->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj, parentidx->dobj.dumpId);
+ // addObjectDependency(&parentidx->dobj, attachinfo[k].dobj.dumpId);
+
addObjectDependency(&attachinfo[k].dobj,
index->indextable->dobj.dumpId);
addObjectDependency(&attachinfo[k].dobj,
--
2.17.0
--fmvA4kSBHQVZhkR6--
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-02-14 20:17 Andres Freund <[email protected]>
0 siblings, 1 reply; 96+ messages in thread
From: Andres Freund @ 2023-02-14 20:17 UTC (permalink / raw)
To: Jonathan S. Katz <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2023-01-23 00:58:40 -0500, Jonathan S. Katz wrote:
> Here is another attempt at this patch that takes into account the SCRAM code
> refactor. I addressed some of Daniel's previous feedback, but will need to
> make another pass on the docs and the assert trace as the main focus of this
> revision was bringing the code inline with the recent changes.
This reliably fails on CI:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3988
I think this is related to encoding issues. The 32bit debian task
intentionally uses LANG=C. Resulting in failures like:
https://api.cirrus-ci.com/v1/artifact/task/6696410851049472/testrun/build-32/testrun/regress/regress...
Windows fails with a similar issue:
https://api.cirrus-ci.com/v1/artifact/task/5676064060473344/testrun/build/testrun/regress/regress/re...
I've set the patch as waiting on author for now.
- Andres
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-02-14 20:19 Jonathan S. Katz <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 96+ messages in thread
From: Jonathan S. Katz @ 2023-02-14 20:19 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2/14/23 3:17 PM, Andres Freund wrote:
> Hi,
>
> On 2023-01-23 00:58:40 -0500, Jonathan S. Katz wrote:
>> Here is another attempt at this patch that takes into account the SCRAM code
>> refactor. I addressed some of Daniel's previous feedback, but will need to
>> make another pass on the docs and the assert trace as the main focus of this
>> revision was bringing the code inline with the recent changes.
>
> This reliably fails on CI:
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3988
>
> I think this is related to encoding issues. The 32bit debian task
> intentionally uses LANG=C. Resulting in failures like:
> https://api.cirrus-ci.com/v1/artifact/task/6696410851049472/testrun/build-32/testrun/regress/regress...
>
> Windows fails with a similar issue:
> https://api.cirrus-ci.com/v1/artifact/task/5676064060473344/testrun/build/testrun/regress/regress/re...
>
> I've set the patch as waiting on author for now.
Thanks for the explanation. I'll work on fixing that in the next go round.
Jonathan
Attachments:
[application/pgp-signature] OpenPGP_signature (840B, ../../[email protected]/2-OpenPGP_signature)
download
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-02-14 23:16 Jonathan S. Katz <[email protected]>
parent: Jonathan S. Katz <[email protected]>
0 siblings, 1 reply; 96+ messages in thread
From: Jonathan S. Katz @ 2023-02-14 23:16 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2/14/23 3:19 PM, Jonathan S. Katz wrote:
> On 2/14/23 3:17 PM, Andres Freund wrote:
>> This reliably fails on CI:
>> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3988
>>
>> I think this is related to encoding issues. The 32bit debian task
>> intentionally uses LANG=C. Resulting in failures like:
>> https://api.cirrus-ci.com/v1/artifact/task/6696410851049472/testrun/build-32/testrun/regress/regress...
>>
>> Windows fails with a similar issue:
>> https://api.cirrus-ci.com/v1/artifact/task/5676064060473344/testrun/build/testrun/regress/regress/re...
>
> Thanks for the explanation. I'll work on fixing that in the next go round.
(First -- I really like the current status of running the tests with
Meson. I'm finding it very easy to use -- doing the locale testing was
pretty easy too!)
I stared at this for a bit to see what we do in other regression tests
using unicode strings. I looked at the regression tests for strings[1]
and ICU collations[2].
In "strings", all the escaped Unicode strings are in the low bits so
they're convertible to ASCII.
In the ICU test, it does a check to see if we're using UTF-8: if we're
not, it bails.
For this patch, the value of the failing test is to ensure that the
SCRAM function honors SASLprep when building the secret. It makes more
sense to use the current character (U+1680), which will be converted to
a space by the algorithm, vs. moving to U+0020 or something that does
not exercise the SASLprep code.
I opted for the approach in [2]. v5 contains the branching logic for the
UTF8 only tests, and the corresponding output files. I tested locally on
macOS against both UTF8 + C locales.
Thanks,
Jonathan
[1]
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/test/regress/sql/strings.sql
[2]
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/test/regress/sql/collate.icu.utf8.s...
From 789fb67e5dec1627a1d20427403cee70184a01f6 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" <[email protected]>
Date: Mon, 31 Oct 2022 16:13:08 -0400
Subject: [PATCH] Add "scram_build_secret" SQL function
This function lets users build SCRAM secrets from SQL
functions and provides the ability for the user to select
the password, salt, and number of iterations for the password
hashing algorithm. Currently this only supports the "sha256"
hash, but can be modified to support additional hashes in the
future.
---
doc/src/sgml/func.sgml | 46 ++++++++++
src/backend/catalog/system_functions.sql | 7 ++
src/backend/libpq/auth-scram.c | 31 +++++--
src/backend/libpq/crypt.c | 2 +-
src/backend/utils/adt/Makefile | 1 +
src/backend/utils/adt/authfuncs.c | 109 +++++++++++++++++++++++
src/backend/utils/adt/meson.build | 1 +
src/include/catalog/pg_proc.dat | 4 +
src/include/libpq/scram.h | 4 +-
src/test/regress/expected/scram.out | 104 +++++++++++++++++++++
src/test/regress/expected/scram_1.out | 93 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/scram.sql | 68 ++++++++++++++
13 files changed, 460 insertions(+), 12 deletions(-)
create mode 100644 src/backend/utils/adt/authfuncs.c
create mode 100644 src/test/regress/expected/scram.out
create mode 100644 src/test/regress/expected/scram_1.out
create mode 100644 src/test/regress/sql/scram.sql
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e09e289a43..a305767e30 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -3513,6 +3513,52 @@ repeat('Pg', 4) <returnvalue>PgPgPgPg</returnvalue>
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>scram_build_secret</primary>
+ </indexterm>
+ <function>scram_build_secret</function> ( <parameter>hash_type</parameter> <type>text</type>
+ , <parameter>password</parameter> <type>text</type>
+ [, <parameter>salt</parameter> <type>bytea</type>
+ [, <parameter>iterations</parameter> <type>integer</type> ] ])
+ <returnvalue>text</returnvalue>
+ </para>
+ <para>
+ Using the values provided in <parameter>hash type</parameter> and
+ <parameter>password</parameter>, builds a SCRAM secret equilvaent to
+ what is stored in
+ <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>rolpassword</structfield>
+ and used with <link linkend="sasl-scram-sha-256"><literal>scram-sha-256</literal></link>
+ authentication. If not provided or set to <literal>NULL</literal>,
+ <parameter>salt</parameter> is randomly generated and
+ <parameter>iterations</parameter> defaults to <literal>4096</literal>.
+ Currently <parameter>hash type</parameter> only supports
+ <literal>sha256</literal>.
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+</programlisting>
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+</programlisting>
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+</programlisting>
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 83ca893444..5a7444df84 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -627,6 +627,13 @@ CREATE OR REPLACE FUNCTION
STABLE PARALLEL SAFE
AS 'sql_localtimestamp';
+-- defaults for building a SCRAM secret
+CREATE OR REPLACE FUNCTION
+ scram_build_secret(text, text, bytea DEFAULT NULL, int DEFAULT NULL)
+RETURNS text
+LANGUAGE INTERNAL
+VOLATILE AS 'scram_build_secret_str';
+
--
-- The default permissions for functions mean that anyone can execute them.
-- A number of functions shouldn't be executable by just anyone, but rather
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4441e0d774..44efffd044 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -468,10 +468,15 @@ scram_exchange(void *opaq, const char *input, int inputlen,
/*
* Construct a SCRAM secret, for storing in pg_authid.rolpassword.
*
+ * "salt_str" can be NULL. If it is, this function will generate a random salt.
+ *
+ * If "iterations" is 0 or less, this function will set it to the default value.
+ *
* The result is palloc'd, so caller is responsible for freeing it.
*/
char *
-pg_be_scram_build_secret(const char *password)
+pg_be_scram_build_secret(const char *password, char *salt_str, int salt_str_len,
+ int iterations, pg_cryptohash_type hash_type)
{
char *prep_password;
pg_saslprep_rc rc;
@@ -488,15 +493,23 @@ pg_be_scram_build_secret(const char *password)
if (rc == SASLPREP_SUCCESS)
password = (const char *) prep_password;
- /* Generate random salt */
- if (!pg_strong_random(saltbuf, SCRAM_DEFAULT_SALT_LEN))
- ereport(ERROR,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random salt")));
+ /* If salt_str is NULL, generate random salt */
+ if (salt_str == NULL)
+ {
+ if (!pg_strong_random(saltbuf, SCRAM_DEFAULT_SALT_LEN))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random salt")));
+ salt_str = saltbuf;
+ salt_str_len = SCRAM_DEFAULT_SALT_LEN;
+ }
+
+ if (iterations <= 0)
+ iterations = SCRAM_DEFAULT_ITERATIONS;
- result = scram_build_secret(PG_SHA256, SCRAM_SHA_256_KEY_LEN,
- saltbuf, SCRAM_DEFAULT_SALT_LEN,
- SCRAM_DEFAULT_ITERATIONS, password,
+ result = scram_build_secret(hash_type, SCRAM_SHA_256_KEY_LEN,
+ salt_str, salt_str_len,
+ iterations, password,
&errstr);
if (prep_password)
diff --git a/src/backend/libpq/crypt.c b/src/backend/libpq/crypt.c
index ef496a0bea..edf94444b6 100644
--- a/src/backend/libpq/crypt.c
+++ b/src/backend/libpq/crypt.c
@@ -140,7 +140,7 @@ encrypt_password(PasswordType target_type, const char *role,
return encrypted_password;
case PASSWORD_TYPE_SCRAM_SHA_256:
- return pg_be_scram_build_secret(password);
+ return pg_be_scram_build_secret(password, NULL, -1, 0, PG_SHA256);
case PASSWORD_TYPE_PLAINTEXT:
elog(ERROR, "cannot encrypt password with 'plaintext'");
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 0de0bbb1b8..7ddb186f96 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -22,6 +22,7 @@ OBJS = \
arraysubs.o \
arrayutils.o \
ascii.o \
+ authfuncs.o \
bool.o \
cash.o \
char.o \
diff --git a/src/backend/utils/adt/authfuncs.c b/src/backend/utils/adt/authfuncs.c
new file mode 100644
index 0000000000..43631abbeb
--- /dev/null
+++ b/src/backend/utils/adt/authfuncs.c
@@ -0,0 +1,109 @@
+/*-------------------------------------------------------------------------
+ *
+ * authfuncs.c
+ * Functions that assist with authentication management
+ *
+ * Portions Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/utils/adt/authfuncs.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "libpq/scram.h"
+#include "utils/builtins.h"
+
+#define SCRAM_BUILD_SECRET_HASH_STR_LEN 6
+
+void parse_cryptohash_type(const char *hash_type_str,
+ pg_cryptohash_type *hash_type);
+
+/*
+ * Build a SCRAM secret that can be used for SCRAM-SHA-256 authentication.
+ *
+ * This function can take four arguments:
+ *
+ * - hash_type_str: the type of hash to use when building the SCRAM secret.
+ * Currently only "sha256" is supported.
+ * - password: a plaintext password. This argument is required. If none of the
+ * other arguments is set, the function short circuits to use a
+ * SCRAM secret generation function that relies on defaults.
+ * - salt_str_enc: a base64 encoded salt. If this is not provided, a salt using
+ * the defaults is generated.
+ * - iterations: the number of iterations to hash the password. If set to 0
+ * or less, the default number of iterations is used.
+ */
+Datum
+scram_build_secret_str(PG_FUNCTION_ARGS)
+{
+ const char *hash_type_str;
+ pg_cryptohash_type hash_type;
+ const char *password;
+ char *salt_str = NULL;
+ int salt_str_len = -1;
+ int iterations = 0;
+ char *secret;
+
+ if (PG_ARGISNULL(0))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
+ errmsg("hash type must not be null")));
+ }
+
+ hash_type_str = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ parse_cryptohash_type(hash_type_str, &hash_type);
+
+ if (PG_ARGISNULL(1))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
+ errmsg("password must not be null")));
+ }
+
+ password = text_to_cstring(PG_GETARG_TEXT_PP(1));
+
+ if (!PG_ARGISNULL(2))
+ {
+ salt_str = text_to_cstring((text *) PG_GETARG_BYTEA_PP(2));
+ salt_str_len = strlen(salt_str);
+ }
+
+ if (!PG_ARGISNULL(3))
+ iterations = PG_GETARG_INT32(3);
+
+ secret = pg_be_scram_build_secret(password, salt_str, salt_str_len,
+ iterations, hash_type);
+
+ Assert(secret != NULL);
+
+ /*
+ * convert the SCRAM secret to text which matches the type for
+ * pg_authid.rolpassword
+ */
+ PG_RETURN_TEXT_P(cstring_to_text(secret));
+}
+
+/*
+ * If "hash_type_str" is a valid cryptohash type that can be used for SCRAM
+ * authetnication, set the value to "hash_type". Otherwise, return an
+ * unsupported error.
+ */
+void
+parse_cryptohash_type(const char *hash_type_str, pg_cryptohash_type *hash_type)
+{
+ pg_cryptohash_type result_type;
+
+ if (pg_strncasecmp(hash_type_str, "sha256", SCRAM_BUILD_SECRET_HASH_STR_LEN) == 0)
+ result_type = PG_SHA256;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("hash not supported: \"%s\"", hash_type_str),
+ errmsg("supported hashes are \"sha256\"")));
+
+ *hash_type = result_type;
+}
diff --git a/src/backend/utils/adt/meson.build b/src/backend/utils/adt/meson.build
index 8515cd9365..52363a2f26 100644
--- a/src/backend/utils/adt/meson.build
+++ b/src/backend/utils/adt/meson.build
@@ -11,6 +11,7 @@ backend_sources += files(
'arraysubs.c',
'arrayutils.c',
'ascii.c',
+ 'authfuncs.c',
'bool.c',
'cash.c',
'char.c',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 66b73c3900..0e5484c8d9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7619,6 +7619,10 @@
{ oid => '3422', descr => 'SHA-512 hash',
proname => 'sha512', proleakproof => 't', prorettype => 'bytea',
proargtypes => 'bytea', prosrc => 'sha512_bytea' },
+{ oid => '8557', descr => 'Build a SCRAM secret',
+ proname => 'scram_build_secret', prorettype => 'text',
+ proisstrict => 'f', proargtypes => 'text text bytea int4',
+ prosrc => 'scram_build_secret_str' },
# crosstype operations for date vs. timestamp and timestamptz
{ oid => '2338',
diff --git a/src/include/libpq/scram.h b/src/include/libpq/scram.h
index b275e1e87e..400f551778 100644
--- a/src/include/libpq/scram.h
+++ b/src/include/libpq/scram.h
@@ -22,7 +22,9 @@
extern PGDLLIMPORT const pg_be_sasl_mech pg_be_scram_mech;
/* Routines to handle and check SCRAM-SHA-256 secret */
-extern char *pg_be_scram_build_secret(const char *password);
+extern char *pg_be_scram_build_secret(const char *password, char *salt_str,
+ int salt_str_len, int iterations,
+ pg_cryptohash_type hash_type);
extern bool parse_scram_secret(const char *secret,
int *iterations,
pg_cryptohash_type *hash_type,
diff --git a/src/test/regress/expected/scram.out b/src/test/regress/expected/scram.out
new file mode 100644
index 0000000000..77313320b5
--- /dev/null
+++ b/src/test/regress/expected/scram.out
@@ -0,0 +1,104 @@
+-- Test building SCRAM functions
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+ERROR: hash type must not be null
+SELECT scram_build_secret('sha256', NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+ERROR: password must not be null
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+ERROR: supported hashes are "sha256"
+SELECT scram_build_secret('sha512', 'password');
+ERROR: supported hashes are "sha256"
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+---------------------------------------------------
+ SCRAM-SHA-256$4096:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+ scram_build_secret
+---------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+ERROR: invalid base64 end sequence
+HINT: Input data is missing padding, is truncated, or is otherwise corrupted.
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+ scram_build_secret
+-------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:YWJj$L27WlKwqjMDY5ZNsyaxGSMii2mhmoUB7xONbxjykmw4=:u1ofGUXUqTbMwfiH+ANWDCpwEjk3j1Xrocy3va/jaCU=
+(1 row)
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+ scram_build_secret
+-----------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+(1 row)
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+ scram_build_secret
+----------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+(1 row)
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+----------------------------------------------------
+ SCRAM-SHA-256$10000:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+ has_default_iterations
+------------------------
+ t
+(1 row)
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- test SASLprep. This tests the case where a user supplies a non-ASCII space
+-- character.
+SELECT
+ scram_build_secret('sha256', U&'one\1680space', decode('h2y81+nUwWp5uIJc4PgyXA==', 'base64')) =
+ 'SCRAM-SHA-256$4096:h2y81+nUwWp5uIJc4PgyXA==$EiywEpO6rM3z3DGehubeoRpp8Orq0XuDUbdT9fQWwz8=:Wh7fq4C+bageihh3vTrkCr7YrlcDTG+JhfcFAuHn/6E=';
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/scram_1.out b/src/test/regress/expected/scram_1.out
new file mode 100644
index 0000000000..27cd91c01d
--- /dev/null
+++ b/src/test/regress/expected/scram_1.out
@@ -0,0 +1,93 @@
+-- Test building SCRAM functions
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+ERROR: hash type must not be null
+SELECT scram_build_secret('sha256', NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+ERROR: password must not be null
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+ERROR: supported hashes are "sha256"
+SELECT scram_build_secret('sha512', 'password');
+ERROR: supported hashes are "sha256"
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+---------------------------------------------------
+ SCRAM-SHA-256$4096:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+ scram_build_secret
+---------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+ERROR: invalid base64 end sequence
+HINT: Input data is missing padding, is truncated, or is otherwise corrupted.
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+ scram_build_secret
+-------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:YWJj$L27WlKwqjMDY5ZNsyaxGSMii2mhmoUB7xONbxjykmw4=:u1ofGUXUqTbMwfiH+ANWDCpwEjk3j1Xrocy3va/jaCU=
+(1 row)
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+ scram_build_secret
+-----------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+(1 row)
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+ scram_build_secret
+----------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+(1 row)
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+----------------------------------------------------
+ SCRAM-SHA-256$10000:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+ has_default_iterations
+------------------------
+ t
+(1 row)
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 15e015b3d6..b360ed07f2 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -28,7 +28,7 @@ test: strings md5 numerology point lseg line box path polygon circle date time t
# geometry depends on point, lseg, line, box, path, polygon, circle
# horology depends on date, time, timetz, timestamp, timestamptz, interval
# ----------
-test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity comments expressions unicode xid mvcc
+test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity comments expressions unicode xid mvcc scram
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/scram.sql b/src/test/regress/sql/scram.sql
new file mode 100644
index 0000000000..be6216f6d3
--- /dev/null
+++ b/src/test/regress/sql/scram.sql
@@ -0,0 +1,68 @@
+-- Test building SCRAM functions
+
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+SELECT scram_build_secret('sha256', NULL);
+SELECT scram_build_secret('sha256', NULL, NULL);
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+SELECT scram_build_secret('sha512', 'password');
+
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- test SASLprep. This tests the case where a user supplies a non-ASCII space
+-- character.
+SELECT
+ scram_build_secret('sha256', U&'one\1680space', decode('h2y81+nUwWp5uIJc4PgyXA==', 'base64')) =
+ 'SCRAM-SHA-256$4096:h2y81+nUwWp5uIJc4PgyXA==$EiywEpO6rM3z3DGehubeoRpp8Orq0XuDUbdT9fQWwz8=:Wh7fq4C+bageihh3vTrkCr7YrlcDTG+JhfcFAuHn/6E=';
--
2.37.1 (Apple Git-137.1)
Attachments:
[text/plain] scram-funcs-v5.patch (26.0K, ../../[email protected]/2-scram-funcs-v5.patch)
download | inline diff:
From 789fb67e5dec1627a1d20427403cee70184a01f6 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" <[email protected]>
Date: Mon, 31 Oct 2022 16:13:08 -0400
Subject: [PATCH] Add "scram_build_secret" SQL function
This function lets users build SCRAM secrets from SQL
functions and provides the ability for the user to select
the password, salt, and number of iterations for the password
hashing algorithm. Currently this only supports the "sha256"
hash, but can be modified to support additional hashes in the
future.
---
doc/src/sgml/func.sgml | 46 ++++++++++
src/backend/catalog/system_functions.sql | 7 ++
src/backend/libpq/auth-scram.c | 31 +++++--
src/backend/libpq/crypt.c | 2 +-
src/backend/utils/adt/Makefile | 1 +
src/backend/utils/adt/authfuncs.c | 109 +++++++++++++++++++++++
src/backend/utils/adt/meson.build | 1 +
src/include/catalog/pg_proc.dat | 4 +
src/include/libpq/scram.h | 4 +-
src/test/regress/expected/scram.out | 104 +++++++++++++++++++++
src/test/regress/expected/scram_1.out | 93 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/scram.sql | 68 ++++++++++++++
13 files changed, 460 insertions(+), 12 deletions(-)
create mode 100644 src/backend/utils/adt/authfuncs.c
create mode 100644 src/test/regress/expected/scram.out
create mode 100644 src/test/regress/expected/scram_1.out
create mode 100644 src/test/regress/sql/scram.sql
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e09e289a43..a305767e30 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -3513,6 +3513,52 @@ repeat('Pg', 4) <returnvalue>PgPgPgPg</returnvalue>
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>scram_build_secret</primary>
+ </indexterm>
+ <function>scram_build_secret</function> ( <parameter>hash_type</parameter> <type>text</type>
+ , <parameter>password</parameter> <type>text</type>
+ [, <parameter>salt</parameter> <type>bytea</type>
+ [, <parameter>iterations</parameter> <type>integer</type> ] ])
+ <returnvalue>text</returnvalue>
+ </para>
+ <para>
+ Using the values provided in <parameter>hash type</parameter> and
+ <parameter>password</parameter>, builds a SCRAM secret equilvaent to
+ what is stored in
+ <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>rolpassword</structfield>
+ and used with <link linkend="sasl-scram-sha-256"><literal>scram-sha-256</literal></link>
+ authentication. If not provided or set to <literal>NULL</literal>,
+ <parameter>salt</parameter> is randomly generated and
+ <parameter>iterations</parameter> defaults to <literal>4096</literal>.
+ Currently <parameter>hash type</parameter> only supports
+ <literal>sha256</literal>.
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+</programlisting>
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+</programlisting>
+ </para>
+ <para>
+ <literal>SELECT scram_build_secret('sha256', 'secret password', decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);</literal>
+ <returnvalue></returnvalue>
+<programlisting>
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+</programlisting>
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 83ca893444..5a7444df84 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -627,6 +627,13 @@ CREATE OR REPLACE FUNCTION
STABLE PARALLEL SAFE
AS 'sql_localtimestamp';
+-- defaults for building a SCRAM secret
+CREATE OR REPLACE FUNCTION
+ scram_build_secret(text, text, bytea DEFAULT NULL, int DEFAULT NULL)
+RETURNS text
+LANGUAGE INTERNAL
+VOLATILE AS 'scram_build_secret_str';
+
--
-- The default permissions for functions mean that anyone can execute them.
-- A number of functions shouldn't be executable by just anyone, but rather
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4441e0d774..44efffd044 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -468,10 +468,15 @@ scram_exchange(void *opaq, const char *input, int inputlen,
/*
* Construct a SCRAM secret, for storing in pg_authid.rolpassword.
*
+ * "salt_str" can be NULL. If it is, this function will generate a random salt.
+ *
+ * If "iterations" is 0 or less, this function will set it to the default value.
+ *
* The result is palloc'd, so caller is responsible for freeing it.
*/
char *
-pg_be_scram_build_secret(const char *password)
+pg_be_scram_build_secret(const char *password, char *salt_str, int salt_str_len,
+ int iterations, pg_cryptohash_type hash_type)
{
char *prep_password;
pg_saslprep_rc rc;
@@ -488,15 +493,23 @@ pg_be_scram_build_secret(const char *password)
if (rc == SASLPREP_SUCCESS)
password = (const char *) prep_password;
- /* Generate random salt */
- if (!pg_strong_random(saltbuf, SCRAM_DEFAULT_SALT_LEN))
- ereport(ERROR,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random salt")));
+ /* If salt_str is NULL, generate random salt */
+ if (salt_str == NULL)
+ {
+ if (!pg_strong_random(saltbuf, SCRAM_DEFAULT_SALT_LEN))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random salt")));
+ salt_str = saltbuf;
+ salt_str_len = SCRAM_DEFAULT_SALT_LEN;
+ }
+
+ if (iterations <= 0)
+ iterations = SCRAM_DEFAULT_ITERATIONS;
- result = scram_build_secret(PG_SHA256, SCRAM_SHA_256_KEY_LEN,
- saltbuf, SCRAM_DEFAULT_SALT_LEN,
- SCRAM_DEFAULT_ITERATIONS, password,
+ result = scram_build_secret(hash_type, SCRAM_SHA_256_KEY_LEN,
+ salt_str, salt_str_len,
+ iterations, password,
&errstr);
if (prep_password)
diff --git a/src/backend/libpq/crypt.c b/src/backend/libpq/crypt.c
index ef496a0bea..edf94444b6 100644
--- a/src/backend/libpq/crypt.c
+++ b/src/backend/libpq/crypt.c
@@ -140,7 +140,7 @@ encrypt_password(PasswordType target_type, const char *role,
return encrypted_password;
case PASSWORD_TYPE_SCRAM_SHA_256:
- return pg_be_scram_build_secret(password);
+ return pg_be_scram_build_secret(password, NULL, -1, 0, PG_SHA256);
case PASSWORD_TYPE_PLAINTEXT:
elog(ERROR, "cannot encrypt password with 'plaintext'");
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 0de0bbb1b8..7ddb186f96 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -22,6 +22,7 @@ OBJS = \
arraysubs.o \
arrayutils.o \
ascii.o \
+ authfuncs.o \
bool.o \
cash.o \
char.o \
diff --git a/src/backend/utils/adt/authfuncs.c b/src/backend/utils/adt/authfuncs.c
new file mode 100644
index 0000000000..43631abbeb
--- /dev/null
+++ b/src/backend/utils/adt/authfuncs.c
@@ -0,0 +1,109 @@
+/*-------------------------------------------------------------------------
+ *
+ * authfuncs.c
+ * Functions that assist with authentication management
+ *
+ * Portions Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/utils/adt/authfuncs.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "libpq/scram.h"
+#include "utils/builtins.h"
+
+#define SCRAM_BUILD_SECRET_HASH_STR_LEN 6
+
+void parse_cryptohash_type(const char *hash_type_str,
+ pg_cryptohash_type *hash_type);
+
+/*
+ * Build a SCRAM secret that can be used for SCRAM-SHA-256 authentication.
+ *
+ * This function can take four arguments:
+ *
+ * - hash_type_str: the type of hash to use when building the SCRAM secret.
+ * Currently only "sha256" is supported.
+ * - password: a plaintext password. This argument is required. If none of the
+ * other arguments is set, the function short circuits to use a
+ * SCRAM secret generation function that relies on defaults.
+ * - salt_str_enc: a base64 encoded salt. If this is not provided, a salt using
+ * the defaults is generated.
+ * - iterations: the number of iterations to hash the password. If set to 0
+ * or less, the default number of iterations is used.
+ */
+Datum
+scram_build_secret_str(PG_FUNCTION_ARGS)
+{
+ const char *hash_type_str;
+ pg_cryptohash_type hash_type;
+ const char *password;
+ char *salt_str = NULL;
+ int salt_str_len = -1;
+ int iterations = 0;
+ char *secret;
+
+ if (PG_ARGISNULL(0))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
+ errmsg("hash type must not be null")));
+ }
+
+ hash_type_str = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ parse_cryptohash_type(hash_type_str, &hash_type);
+
+ if (PG_ARGISNULL(1))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
+ errmsg("password must not be null")));
+ }
+
+ password = text_to_cstring(PG_GETARG_TEXT_PP(1));
+
+ if (!PG_ARGISNULL(2))
+ {
+ salt_str = text_to_cstring((text *) PG_GETARG_BYTEA_PP(2));
+ salt_str_len = strlen(salt_str);
+ }
+
+ if (!PG_ARGISNULL(3))
+ iterations = PG_GETARG_INT32(3);
+
+ secret = pg_be_scram_build_secret(password, salt_str, salt_str_len,
+ iterations, hash_type);
+
+ Assert(secret != NULL);
+
+ /*
+ * convert the SCRAM secret to text which matches the type for
+ * pg_authid.rolpassword
+ */
+ PG_RETURN_TEXT_P(cstring_to_text(secret));
+}
+
+/*
+ * If "hash_type_str" is a valid cryptohash type that can be used for SCRAM
+ * authetnication, set the value to "hash_type". Otherwise, return an
+ * unsupported error.
+ */
+void
+parse_cryptohash_type(const char *hash_type_str, pg_cryptohash_type *hash_type)
+{
+ pg_cryptohash_type result_type;
+
+ if (pg_strncasecmp(hash_type_str, "sha256", SCRAM_BUILD_SECRET_HASH_STR_LEN) == 0)
+ result_type = PG_SHA256;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("hash not supported: \"%s\"", hash_type_str),
+ errmsg("supported hashes are \"sha256\"")));
+
+ *hash_type = result_type;
+}
diff --git a/src/backend/utils/adt/meson.build b/src/backend/utils/adt/meson.build
index 8515cd9365..52363a2f26 100644
--- a/src/backend/utils/adt/meson.build
+++ b/src/backend/utils/adt/meson.build
@@ -11,6 +11,7 @@ backend_sources += files(
'arraysubs.c',
'arrayutils.c',
'ascii.c',
+ 'authfuncs.c',
'bool.c',
'cash.c',
'char.c',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 66b73c3900..0e5484c8d9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7619,6 +7619,10 @@
{ oid => '3422', descr => 'SHA-512 hash',
proname => 'sha512', proleakproof => 't', prorettype => 'bytea',
proargtypes => 'bytea', prosrc => 'sha512_bytea' },
+{ oid => '8557', descr => 'Build a SCRAM secret',
+ proname => 'scram_build_secret', prorettype => 'text',
+ proisstrict => 'f', proargtypes => 'text text bytea int4',
+ prosrc => 'scram_build_secret_str' },
# crosstype operations for date vs. timestamp and timestamptz
{ oid => '2338',
diff --git a/src/include/libpq/scram.h b/src/include/libpq/scram.h
index b275e1e87e..400f551778 100644
--- a/src/include/libpq/scram.h
+++ b/src/include/libpq/scram.h
@@ -22,7 +22,9 @@
extern PGDLLIMPORT const pg_be_sasl_mech pg_be_scram_mech;
/* Routines to handle and check SCRAM-SHA-256 secret */
-extern char *pg_be_scram_build_secret(const char *password);
+extern char *pg_be_scram_build_secret(const char *password, char *salt_str,
+ int salt_str_len, int iterations,
+ pg_cryptohash_type hash_type);
extern bool parse_scram_secret(const char *secret,
int *iterations,
pg_cryptohash_type *hash_type,
diff --git a/src/test/regress/expected/scram.out b/src/test/regress/expected/scram.out
new file mode 100644
index 0000000000..77313320b5
--- /dev/null
+++ b/src/test/regress/expected/scram.out
@@ -0,0 +1,104 @@
+-- Test building SCRAM functions
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+ERROR: hash type must not be null
+SELECT scram_build_secret('sha256', NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+ERROR: password must not be null
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+ERROR: supported hashes are "sha256"
+SELECT scram_build_secret('sha512', 'password');
+ERROR: supported hashes are "sha256"
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+---------------------------------------------------
+ SCRAM-SHA-256$4096:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+ scram_build_secret
+---------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+ERROR: invalid base64 end sequence
+HINT: Input data is missing padding, is truncated, or is otherwise corrupted.
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+ scram_build_secret
+-------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:YWJj$L27WlKwqjMDY5ZNsyaxGSMii2mhmoUB7xONbxjykmw4=:u1ofGUXUqTbMwfiH+ANWDCpwEjk3j1Xrocy3va/jaCU=
+(1 row)
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+ scram_build_secret
+-----------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+(1 row)
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+ scram_build_secret
+----------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+(1 row)
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+----------------------------------------------------
+ SCRAM-SHA-256$10000:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+ has_default_iterations
+------------------------
+ t
+(1 row)
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- test SASLprep. This tests the case where a user supplies a non-ASCII space
+-- character.
+SELECT
+ scram_build_secret('sha256', U&'one\1680space', decode('h2y81+nUwWp5uIJc4PgyXA==', 'base64')) =
+ 'SCRAM-SHA-256$4096:h2y81+nUwWp5uIJc4PgyXA==$EiywEpO6rM3z3DGehubeoRpp8Orq0XuDUbdT9fQWwz8=:Wh7fq4C+bageihh3vTrkCr7YrlcDTG+JhfcFAuHn/6E=';
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/scram_1.out b/src/test/regress/expected/scram_1.out
new file mode 100644
index 0000000000..27cd91c01d
--- /dev/null
+++ b/src/test/regress/expected/scram_1.out
@@ -0,0 +1,93 @@
+-- Test building SCRAM functions
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+ERROR: hash type must not be null
+SELECT scram_build_secret('sha256', NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL);
+ERROR: password must not be null
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+ERROR: password must not be null
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+ERROR: supported hashes are "sha256"
+SELECT scram_build_secret('sha512', 'password');
+ERROR: supported hashes are "sha256"
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+---------------------------------------------------
+ SCRAM-SHA-256$4096:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+ scram_build_secret
+---------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:MTIzNDU2Nzg5MGFiY2RlZg==$D5BmucT796UQKargx2k3fdqjDYR7cH/L0viKKhGo6kA=:M33+iHFOESP8C3DKLDb2k5QAhkNVWEbp/YUIFd2CxN4=
+(1 row)
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+ERROR: invalid base64 end sequence
+HINT: Input data is missing padding, is truncated, or is otherwise corrupted.
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+ scram_build_secret
+-------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:YWJj$L27WlKwqjMDY5ZNsyaxGSMii2mhmoUB7xONbxjykmw4=:u1ofGUXUqTbMwfiH+ANWDCpwEjk3j1Xrocy3va/jaCU=
+(1 row)
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+ scram_build_secret
+-----------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$4096:q7pUMg==$05Nb9QHwHkMA0CRcYaEfwtgZ+3kStIefz8fLMjTEtio=:P126h1ycyP938E69yxktEfhoAILbiwL/UMsMk3Efb6o=
+(1 row)
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+ scram_build_secret
+----------------------------------------------------------------------------------------------------------------------------------------
+ SCRAM-SHA-256$10000:MTIzNDU2Nzg5MGFiY2RlZg==$9NkDu1TFpx3L30zMgHUqjRNSq3GRZRrdWU4TuGOnT3Q=:svuIH9L6HH8loyKWguT64XXoOLCrr4FkVViPd2JVR4M=
+(1 row)
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+ scram_secret
+----------------------------------------------------
+ SCRAM-SHA-256$10000:<salt>$<storedkey>:<serverkey>
+(1 row)
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+ has_default_iterations
+------------------------
+ t
+(1 row)
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 15e015b3d6..b360ed07f2 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -28,7 +28,7 @@ test: strings md5 numerology point lseg line box path polygon circle date time t
# geometry depends on point, lseg, line, box, path, polygon, circle
# horology depends on date, time, timetz, timestamp, timestamptz, interval
# ----------
-test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity comments expressions unicode xid mvcc
+test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity comments expressions unicode xid mvcc scram
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/scram.sql b/src/test/regress/sql/scram.sql
new file mode 100644
index 0000000000..be6216f6d3
--- /dev/null
+++ b/src/test/regress/sql/scram.sql
@@ -0,0 +1,68 @@
+-- Test building SCRAM functions
+
+-- test all nulls
+-- fail
+SELECT scram_build_secret(NULL, NULL);
+SELECT scram_build_secret('sha256', NULL);
+SELECT scram_build_secret('sha256', NULL, NULL);
+SELECT scram_build_secret('sha256', NULL, NULL, NULL);
+
+-- test unsupported hashes
+-- fail
+SELECT scram_build_secret('sha384', 'password');
+SELECT scram_build_secret('sha512', 'password');
+
+-- generated a SCRAM secret from a plaintext password
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password'),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]+)\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+
+-- test building a SCRAM secret with a predefined salt with a valid base64
+-- encoded string
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'));
+
+-- test building a SCRAM secret with a predefined salt that is not a valid
+-- base64 string
+-- fail
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('abc', 'base64'));
+
+-- test building a SCRAM secret with a salt that looks like a string but is
+-- cast to a bytea
+SELECT scram_build_secret('sha256', 'secret password', 'abc');
+
+-- test building a SCRAM secret with a bytea salt using the hex format
+SELECT scram_build_secret('sha256', 'secret password', '\xabba5432');
+
+-- test building a SCRAM secret with a valid salt and a different set of
+-- iterations
+SELECT scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), 10000);
+
+-- test what happens when the salt is a NULL value.
+-- this should build a SCRAM secret using a random salt.
+SELECT regexp_replace(
+ scram_build_secret('sha256', 'secret password', NULL, 10000),
+ '(SCRAM-SHA-256)\$(\d+):([a-zA-Z0-9+/=]{24})\$([a-zA-Z0-9+=/]+):([a-zA-Z0-9+/=]+)',
+ '\1$\2:<salt>$<storedkey>:<serverkey>') AS scram_secret;
+
+-- test what happens when iterations is a null value. this should set iterations
+-- to be the default, currently 4096.
+SELECT
+ scram_build_secret('sha256', 'secret password',
+ decode('MTIzNDU2Nzg5MGFiY2RlZg==', 'base64'), NULL) ~
+ '^SCRAM-SHA-256\$4096' AS has_default_iterations;
+
+-- skip the remaining tests if this not a UTF8 server
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- test SASLprep. This tests the case where a user supplies a non-ASCII space
+-- character.
+SELECT
+ scram_build_secret('sha256', U&'one\1680space', decode('h2y81+nUwWp5uIJc4PgyXA==', 'base64')) =
+ 'SCRAM-SHA-256$4096:h2y81+nUwWp5uIJc4PgyXA==$EiywEpO6rM3z3DGehubeoRpp8Orq0XuDUbdT9fQWwz8=:Wh7fq4C+bageihh3vTrkCr7YrlcDTG+JhfcFAuHn/6E=';
--
2.37.1 (Apple Git-137.1)
[application/pgp-signature] OpenPGP_signature (840B, ../../[email protected]/3-OpenPGP_signature)
download
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-03-22 06:48 Michael Paquier <[email protected]>
parent: Jonathan S. Katz <[email protected]>
0 siblings, 1 reply; 96+ messages in thread
From: Michael Paquier @ 2023-03-22 06:48 UTC (permalink / raw)
To: Jonathan S. Katz <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 14, 2023 at 06:16:18PM -0500, Jonathan S. Katz wrote:
> I opted for the approach in [2]. v5 contains the branching logic for the
> UTF8 only tests, and the corresponding output files. I tested locally on
> macOS against both UTF8 + C locales.
I was reading this thread again, and pondered on this particular
point:
https://www.postgresql.org/message-id/[email protected]...
We've had our share of complains over the years that Postgres logs
password data in the logs with various DDLs, so I'd tend to agree that
this is not a practice we should try to encourage more. The
parameterization of the SCRAM verifiers through GUCs (like Daniel's
https://commitfest.postgresql.org/42/4201/ for the iteration number)
is more promising because it is possible to not have to send the
password over the wire with once we let libpq take care of the
computation, and the server would not know about that.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-03-22 14:06 Jonathan S. Katz <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 96+ messages in thread
From: Jonathan S. Katz @ 2023-03-22 14:06 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/22/23 2:48 AM, Michael Paquier wrote:
> On Tue, Feb 14, 2023 at 06:16:18PM -0500, Jonathan S. Katz wrote:
>> I opted for the approach in [2]. v5 contains the branching logic for the
>> UTF8 only tests, and the corresponding output files. I tested locally on
>> macOS against both UTF8 + C locales.
>
> I was reading this thread again, and pondered on this particular
> point:
> https://www.postgresql.org/message-id/[email protected]...
>
> We've had our share of complains over the years that Postgres logs
> password data in the logs with various DDLs, so I'd tend to agree that
> this is not a practice we should try to encourage more. The
> parameterization of the SCRAM verifiers through GUCs (like Daniel's
> https://commitfest.postgresql.org/42/4201/ for the iteration number)
> is more promising because it is possible to not have to send the
> password over the wire with once we let libpq take care of the
> computation, and the server would not know about that
I generally agree with not allowing password data to be in logs, but in
practice, there are a variety of tools and extensions that obfuscate or
remove passwords from PostgreSQL logs. Additionally, this function is
not targeted for SQL statements directly, but stored procedures.
For example, an extension like "pg_tle" exposes the ability for someone
to write a "check_password_hook" directly from PL/pgSQL[1] (and other
languages). As we've made it a best practice to pre-hash the password on
the client-side, a user who wants to write a check password hook against
a SCRAM verifier needs to be able to compare the verifier against some
existing set of plaintext criteria, and has to write their own function
to do it. I have heard several users who have asked to do this, and the
only feedback I can give them is "implement your own SCRAM build secret
function."
And, if my PostgreSQL server _is_ the client, e.g. it's making a dblink
call to another PostgreSQL server, the only way it can modify a password
is by sending the plaintext credential over the wire.
I don't see how the parameterization work applies here -- would we allow
salts to be parameterized? -- and it still would not allow the server to
build out a SCRAM secret for these cases.
Maybe I'm not conveying the problem this is solving -- I'm happy to go
one more round trying to make it clearer -- but if this is not clear,
it'd be good to at develop an alternative approach to this before
withdrawing the patch.
Thanks,
Jonathan
[1]
https://github.com/aws/pg_tle/blob/main/docs/06_plpgsql_examples.md#example-password-check-hook-agai...
Attachments:
[application/pgp-signature] OpenPGP_signature (840B, ../../[email protected]/2-OpenPGP_signature)
download
^ permalink raw reply [nested|flat] 96+ messages in thread
* Re: User functions for building SCRAM secrets
@ 2023-03-24 15:47 Daniel Gustafsson <[email protected]>
parent: Jonathan S. Katz <[email protected]>
0 siblings, 0 replies; 96+ messages in thread
From: Daniel Gustafsson @ 2023-03-24 15:47 UTC (permalink / raw)
To: Jonathan S. Katz <[email protected]>; +Cc: Michael Paquier <[email protected]>; Andres Freund <[email protected]>; PostgreSQL Hackers <[email protected]>
> On 22 Mar 2023, at 15:06, Jonathan S. Katz <[email protected]> wrote:
> On 3/22/23 2:48 AM, Michael Paquier wrote:
>> I was reading this thread again, and pondered on this particular
>> point:
>> https://www.postgresql.org/message-id/[email protected]...
>> We've had our share of complains over the years that Postgres logs
>> password data in the logs with various DDLs, so I'd tend to agree that
>> this is not a practice we should try to encourage more. The
>> parameterization of the SCRAM verifiers through GUCs (like Daniel's
>> https://commitfest.postgresql.org/42/4201/ for the iteration number)
>> is more promising because it is possible to not have to send the
>> password over the wire with once we let libpq take care of the
>> computation, and the server would not know about that
>
> I generally agree with not allowing password data to be in logs, but in practice, there are a variety of tools and extensions that obfuscate or remove passwords from PostgreSQL logs. Additionally, this function is not targeted for SQL statements directly, but stored procedures.
>
> For example, an extension like "pg_tle" exposes the ability for someone to write a "check_password_hook" directly from PL/pgSQL[1] (and other languages). As we've made it a best practice to pre-hash the password on the client-side, a user who wants to write a check password hook against a SCRAM verifier needs to be able to compare the verifier against some existing set of plaintext criteria, and has to write their own function to do it. I have heard several users who have asked to do this, and the only feedback I can give them is "implement your own SCRAM build secret function."
I'm not sure I follow; doesn't this - coupled with this patch - imply passing
the plaintext password from client to the server, hashing it with a known salt
and comparing this with something in plaintext hashed with the same known salt?
If so, that's admittedly not a usecase I am terribly excited about. My
preference is to bring password checks to the plaintext password, not bring the
plaintext password to the password check.
> And, if my PostgreSQL server _is_ the client, e.g. it's making a dblink call to another PostgreSQL server, the only way it can modify a password is by sending the plaintext credential over the wire.
My experience with dblink setups is too small to have much insight here, but I
(perhaps naively) assumed that dblink setups generally involved two servers
under the same control making such changes be possible out of band.
> Maybe I'm not conveying the problem this is solving -- I'm happy to go one more round trying to make it clearer -- but if this is not clear, it'd be good to at develop an alternative approach to this before withdrawing the patch.
If this is mainly targeting extension use, is there a way in which an extension
could have all this functionality with no: a) not exposing any of it from the
server; b) not having to copy/paste lots into the extension and; c) have a
reasonable way to keep up with any changes made in the backend?
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 96+ messages in thread
end of thread, other threads:[~2023-03-24 15:47 UTC | newest]
Thread overview: 96+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v10 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v7 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v6 7/7] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v9 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2020-11-25 23:34 [PATCH v8 8/8] pg_dump: partitioned index depend on its partitions Justin Pryzby <[email protected]>
2023-02-14 20:17 Re: User functions for building SCRAM secrets Andres Freund <[email protected]>
2023-02-14 20:19 ` Re: User functions for building SCRAM secrets Jonathan S. Katz <[email protected]>
2023-02-14 23:16 ` Re: User functions for building SCRAM secrets Jonathan S. Katz <[email protected]>
2023-03-22 06:48 ` Re: User functions for building SCRAM secrets Michael Paquier <[email protected]>
2023-03-22 14:06 ` Re: User functions for building SCRAM secrets Jonathan S. Katz <[email protected]>
2023-03-24 15:47 ` Re: User functions for building SCRAM secrets Daniel Gustafsson <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox