public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
20+ messages / 2 participants
[nested] [flat]
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables
@ 2020-08-14 17:26 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Alvaro Herrera @ 2020-08-14 17:26 UTC (permalink / raw)
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Reported-by: Tom Lane <[email protected]>
Author: �lvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_dump/pg_dump.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2cb3f9b083 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -235,6 +235,7 @@ static DumpableObject *createBoundaryObjects(void);
static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
+static void addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo);
@@ -7517,25 +7518,20 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
reftable = findTableByOid(constrinfo[j].confrelid);
if (reftable && reftable->relkind == RELKIND_PARTITIONED_TABLE)
{
- IndxInfo *refidx;
Oid indexOid = atooid(PQgetvalue(res, j, i_conindid));
if (indexOid != InvalidOid)
{
for (int k = 0; k < reftable->numIndexes; k++)
{
- SimplePtrListCell *cell;
+ IndxInfo *refidx;
/* not our index? */
if (reftable->indexes[k].dobj.catId.oid != indexOid)
continue;
refidx = &reftable->indexes[k];
- for (cell = refidx->partattaches.head; cell;
- cell = cell->next)
- addObjectDependency(&constrinfo[j].dobj,
- ((DumpableObject *)
- cell->ptr)->dumpId);
+ addConstrChildIdxDeps(&constrinfo[j].dobj, refidx);
break;
}
}
@@ -7548,6 +7544,35 @@ getConstraints(Archive *fout, TableInfo tblinfo[], int numTables)
destroyPQExpBuffer(query);
}
+/*
+ * addConstrChildIdxDeps
+ *
+ * Recursive subroutine for getConstraints
+ *
+ * Given an object representing a foreign key constraint and an index on the
+ * partitioned table it references, mark the constraint object as dependent
+ * on the DO_INDEX_ATTACH object of each index partition, recursively
+ * drilling down to their partitions if any. This ensures that the FK is not
+ * restored until the index is fully marked valid.
+ */
+static void
+addConstrChildIdxDeps(DumpableObject *dobj, IndxInfo *refidx)
+{
+ SimplePtrListCell *cell;
+
+ Assert(dobj->objType == DO_FK_CONSTRAINT);
+
+ for (cell = refidx->partattaches.head; cell; cell = cell->next)
+ {
+ IndexAttachInfo *attach = (IndexAttachInfo *) cell->ptr;
+
+ addObjectDependency(dobj, attach->dobj.dumpId);
+
+ if (attach->partitionIdx->partattaches.head != NULL)
+ addConstrChildIdxDeps(dobj, attach->partitionIdx);
+ }
+}
+
/*
* getDomainConstraints
*
--
2.20.1
--azLHFNyN32YCQGCU--
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Question about non-blocking mode in libpq
@ 2023-10-31 16:58 Bruce Momjian <[email protected]>
0 siblings, 0 replies; 20+ messages in thread
From: Bruce Momjian @ 2023-10-31 16:58 UTC (permalink / raw)
To: Yugo NAGATA <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers
On Wed, Jul 21, 2021 at 10:15:09AM +0900, Yugo NAGATA wrote:
> I understood that, although PQgetResult() also flushes the buffer, we still
> should call PQflush() beforehand because we would not like get blocked after
> calling PQgetResult(). Thanks.
I modified your patch, attached, that I would like to apply to all
supported versions.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Only you can decide what is important to you.
Attachments:
[text/x-diff] block.diff (741B, ../../[email protected]/2-block.diff)
download | inline diff:
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 64b2910fee..d1609ddd9a 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5319,8 +5319,8 @@ int PQsetnonblocking(PGconn *conn, int arg);
In the nonblocking state, calls to
<xref linkend="libpq-PQsendQuery"/>, <xref linkend="libpq-PQputline"/>,
<xref linkend="libpq-PQputnbytes"/>, <xref linkend="libpq-PQputCopyData"/>,
- and <xref linkend="libpq-PQendcopy"/> will not block but instead return
- an error if they need to be called again.
+ and <xref linkend="libpq-PQendcopy"/> will not block; their changes
+ are stored in the local output buffer until they are flushed.
</para>
<para>
^ permalink raw reply [nested|flat] 20+ messages in thread
end of thread, other threads:[~2023-10-31 16:58 UTC | newest]
Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2020-08-14 17:26 [PATCH v2] pg_dump: fix dependencies on FKs to multi-level partitioned tables Alvaro Herrera <[email protected]>
2023-10-31 16:58 Re: Question about non-blocking mode in libpq Bruce Momjian <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox